home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
Cream of the Crop 20
/
Cream of the Crop 20 (Terry Blount) (1996).iso
/
bbs
/
ulp_222.zip
/
ULP.DOC
< prev
next >
Wrap
Text File
|
1996-07-13
|
82KB
|
1,631 lines
┌───────────────────┐
│ │ ║ UpLoadProcessor
│ ╥ ╥ ╥ │ ║
│ ║ ║ ║ ╓──╖ │ ║ Version 2.22
│ ║ ║ ║ ║ ║ │ ║
│ ╙───╜ ╨ ║──╜ │ ║ (c) Copyright 1992-1996 - Stacy Smith
│ ╨ │ ║
└───────────────────┘ ║
════════════════════╝
┌───────────────────────────────────────────────────────────────────────┐
│ Winner of the 1995 PCBoard Favorite Add-On Contest, Overall │
│ Winner of the 1994 PCBoard Favorite Add-On Contest, Other Utilities │
└───────────────────────────────────────────────────────────────────────┘
Courtesy of:
The Bloom Beacon-Picayune BBS
Node 1: *** DOWN TEMPORARILY ***
FidoNet
ILink
Intelec
Stacy Smith
305 Cottonwood Lane
Holly Springs, NC 27540
┌───────────────────┐
│ 1. Introduction │
└───────────────────┘
This system was born out of a need for a universal upload processor. There are
many alternative systems available, but they are limited to the ZIP format and
perhaps one or two others. Few are able to handle self-extracting archives.
Most are limited in the number of levels of archive nesting allowed in a file
to be tested. Very few properly handle archives with imbedded paths. Many
require the use of a third-party duplicate file checking system if you want to
screen your uploads for duplicates.
Tired of waiting for PKZIP 2.something-or-another, I converted my BBS files
over to the ARJ compression format, due to its superior compression ratio over
PKZip and its features over LHA (I have since switched back to ZIP...the
undertow was overwhelming <g>). But due to that decision, the need for a
universal upload processor became apparent, so off I went.
While I was at it, I decided to incorporate other technologies, such as
duplicate checking, archive format conversion, customized displays and
comments, internationalization, foreign language support, information lines,
internal description files, etc., into a single package. This software is the
result of my efforts to allow my BBS to handle any archive that my users can
throw at it.
┌─────────────────────────────────────────────┐
│ 2. Features of the UpLoadProcessor System │
└─────────────────────────────────────────────┘
∙ Native versions available for both 16-bit DOS and 32-bit OS/2! DOS version
is fully OS/2, DESQview and Windows aware, including time-slice releasing.
∙ Identifies and processes ARC, ARJ, HYP, LZH, PAK, RAR, SQZ, ZIP, ZOO, GIF,
JPEG and BMP files, regardless of their file extensions (ideal for software
distribution network files such as SDN).
∙ Identifies and processes ARJ, LZH, RAR, SQZ and ZIP self-extracting (SFX)
archives.
∙ Detects, processes and converts any archive with imbedded paths, retaining
all path information.
∙ Scans ARC, PAK, ZIP and ZIP SFX archives for DOS reserved keywords to
prevent hacking by hex-editing. (ARJ and LHA are resistant to these type of
hacking attempts).
∙ Detects ARJ security envelopes and ZIP version 1.x and 2.x authenticity
verification (-AV) stamps and may be retained intact.
∙ Detects and rejects encrypted ARJ and ZIP archives.
∙ Uses a recursive processing routine that will allow (theoretically)
unlimited nested archives and paths (the only limit is imposed by the OS
path). This routine has been tested to 7 levels deep as of this writing.
∙ Selected uncompressed files uploaded can be processed and compressed using
your default format.
∙ Uploads can be filtered, privileged and TCANned on a wide variety of
criteria, including filename, file size, description keywords, etc..
∙ Removes known BBS ads from archives; includes BBS ads database maintenance
functionality so that sysops can update their BBS ads databases in real
time. ULP can also insert a BBS ad (ugh), if desired.
∙ Sends messages to the online user and/or sysop for failed and/or passed
uploads, fully customizable and with multi-lingual support.
∙ Updates the PCBoard DOWNLOAD.TXT file, if desired, with the correct archive
extension to reflect the conversion process.
∙ Allows the use of up to 10 different archiving programs, all user-
configurable. Any archiving program used that is not listed above will be
identified using its unique file extension only, until its signature is
determined and incorporated into ULP (after a new archive has demonstrated
widespread use).
∙ Archive comments can be customized on an archive-by-archive basis through
the use of a template and @-variables.
∙ Allows the use of up to 10 different file-checking programs, all user-
configurable, for virus and trojan checking, third-party utilities, etc.
∙ Rejects GIF and JPEG files based upon image width, height, colors and/or
compression. Different values can be selected for GIF and JPEG.
∙ Allows the use of GIF and JPEG file checking programs, completely and
individually configurable.
∙ User-definable disposition (keep, rename or delete) of corrupted, duplicate
or other archives (not virus-related).
∙ User-definable disposition of virus-infected archives.
∙ User-definable disposition of complete duplicate archives.
∙ Incorporates it's own duplicate checking system, as well as the associated
database processing software. This internal system is extremely fast and
it's database is much smaller than other systems. Despite it's size, the
possibility of a false duplication is almost 1 in 10 trillion! The system
is self-validating, to quickly determine if a database has been corrupted
or altered.
∙ Optional seamless interface with the ZDCS duplication system.
∙ ULP determines duplication using two filters, total duplication and
executable duplication, preventing false rejection by simply counting up
the number of blatantly duplicate files as other duplication systems do.
∙ Converts all uploads into a default archive format of your choosing, or
they may be re-archived in their original format (user-defined). Nested
and pathed archives can also be converted to your default format, or
re-archived using their original format. SFX archives can be archived
using your default format, or optionally left alone after verification.
Archiving formats can be individually configured to not be converted to
your default, effectively allowing multiple defaults.
∙ Can utilize a user-defined time window (in months) for acceptance of new
upload files, based on the actual or statistical average age of the files
within the archive, or optionally, the age of the newest file.
∙ Supports the use of private and public upload directories. Moves files and
upload descriptions from the private directory to the public directory.
Rejected uploads can be optionally moved to an offline area of your
choosing. Single directory area operation may also be configured.
∙ Duplication and age limits can be set on an area-by-area basis.
∙ Honors the '/' identifier in the description marking the file as a private
upload for the sysop by processing the file, but not making it public.
∙ Supports the use of DESC.SDI, FILE-ID.DIZ and VENDINFO.DIZ description
files in an archive, user-configurable. The number of lines inserted is
also configurable, up to 40 lines maximum.
∙ Smart word-wrapping word-wraps descriptions or leaves them as entered,
depending upon the presence of boxes, etc.
∙ Can optionally insert an archive or GIF/JPEG information line in the file
description that contains various information about the upload.
∙ Three modes of online testing are available: slow mode, which completely
processes files at the time of upload; normal mode, which fully unpacks the
archive and tests each file individually; and fast mode, which scans a ZIP
or ARJ archive directly for file CRCs, sizes and dates, and uses the
archiving program's internal integrity testing.
∙ The online tester will accept a redirected ARJ or PKUNZIP archive listing
file to pre-verify the duplication and age limits before a user uploads the
actual archive, saving him or her wailing and gnashing of teeth.
∙ ULP can generate COM port status output to inform the online user of the
progress of testing. The format of the screen display is completely
sysop-designed through the use of template files; different template files
can be used for high-speed and low-speed callers. Multi-lingual templates
files are supported. ULP supports IRQs 2 through 15, non-standard port
addresses and baud rates to 115K in direct mode, supports FOSSIL drivers
and the OS/2 SIO API interface. The port information can be defined on the
command-line or can be read directly from PCBOARD.SYS or DOOR.SYS.
∙ Import of FWKCS(TM) contents_signature databases is supported to ease
transition to the ULP duplication system. No need to rebuild databases for
FWKCS users!
∙ Archives failed for exceeding duplication limits can be viewed using ULPSM,
to allow easy determination of manual archive acceptance.
∙ User-selectable process logging to a disk file. Two logging modes are
available: terse and verbose. A debug operational mode can also be
utilized to help track down configuration errors.
∙ Menu-driven windowed system manager for maximum ease of configuration,
including context-sensitive help. Automatic installation speeds the
configuration and setup process for new users.
∙ Written mostly in C (and a little assembler) for optimal speed, using
Microsoft C/C++ v7 and Watcom C/C++ v10.
┌─────────────────────────────────────────────────────┐
│ 3. Files Included in the ULP Distribution Archive │
└─────────────────────────────────────────────────────┘
ULP.EXE Upload processing program.
ULPSM.EXE System and configuration manager for the ULP system.
ULPSM.HLP Help data file for the ULP system.
BBSADS.DB BBS ads database file.
ULP.DOC This file.
FAQ.DOC Frequently asked questions list.
SUPPORT.DOC List of authorized support sites for my shareware.
HISTORY.DOC ULP revision history in reverse order.
UPGRADE.DOC Information specific to upgrading from previous versions.
REGISTER.FRM Registration form for ULP and other software.
FILE_ID.DIZ Internal description file.
When you unzip the distribution archive, you should see my PKZIP authenticity
verification stamp, and a '-AV' after every file in the archive:
# SSU301 The Bloom Beacon-Picayune BBS
If there are any files missing or added, or the -AV stamp is missing, the
archive may have been tampered with. It would be advisable to call my BBS
(listed at the top of this document) or one of the support sites listed in
SUPPORT.DOC for the latest version of ULP.
The 32-bit OS/2 executables ULP2.EXE and ULPSM2.EXE are distributed in a
separate archive named ULP2_xxx.ZIP, where the "xxx" is the revision level of
ULP. Since this archive contains only the OS/2 executables, the general
distribution archive must be downloaded as well. This was done to reduce the
size of the general distribution archive for those sysops not running OS/2 as
their operating system.
┌───────────────────────────┐
│ 4. Program Requirements │
└───────────────────────────┘
To the best of my knowledge, the ULP programs will run on most any machine
capable of running PCBoard 14.5+. My BBS setup was OS/2 and DOS/DESQview on a
LANtastic network with hard disks and CD-ROMs, but other sysops that I have
been in contact with have successfully implemented ULP on setups with other
varying hardware and software.
ULP has been developed and tested using the following third party utilities:
ARJ 2.10 and higher (by Robert Jung)
HYPER 2.5 (by P. Sawatzki and K. P. Nischke)
LHA 2.12 and higher (by Haruyasu Yoshizaki)
LHarc 1.13c (by Haruyasu Yoshizaki)
PAK 2.51 (by NoGate Consulting)
PKPAK 3.61 (by PKWare)
PKZIP 1.10 and higher (by PKWare)
RAR 1.53 and higher [both DOS and OS/2 versions] (by Eugene Roshal)
SQZ 1.08.2 (by Jonas Hammarberg)
ZOO 2.01 and higher (by Rahul Deshi)
AntiAd 0.98ß and higher (by Stacy Smith)
F-PROT 2.07 and higher (by Frisk Software International)
SCAN V82 and higher (by McAfee Associates)
GFXCheck 1.00 and higher (by Stacy Smith)
GIFTEST 4.0 Beta 10 and higher (by Max Bernard/Dave Navarro)
ZDCS 2.03 and higher (by Stacy Smith)
TXT2MSG 2.42 and higher (by Robert Vostreys)
SHROOM 2.3a (by Davis Augustine)
UnZip 5.12 (by Info-ZIP)
ZIP 2.0.1 (by Adler, Wales, Gailly and Rommel)
OS/2Scan 2.2.0 (by McAfee Associates)
PipeDOS (by Scott Maxwell)
The ULP system requires DOS 3.x or later (or compatible, such as an OS/2 DOS
VDM), as it uses DOS SHARE-compatible file reads and writes, and can use the
DOS PATH to find the archiving and other utilities. The 32-bit OS/2 version of
ULP must be run under OS/2 2.0 or later.
ULP's memory requirements are relatively small (about 250K for ULP.EXE, 450K
for ULPSM.EXE), and can easily live in the same amount of memory as PCBoard
(450K as recommended by Clark Development). Note that all programs are spawned
or shelled, which reduces the free memory for the program being executed. It
would be a good idea to have as much free conventional memory as possible (ULP
itself cannot use EMS or XMS memory except for swapping), especially if you use
the ARJ compression system, which requires more than 300K itself to run. None
of these limits exist for the 32-bit OS/2 versions of ULP, since we are using a
real operating system. <g>
┌───────────────────┐
│ 5. Registration │
└───────────────────┘
The ULP system is not free; nor is ULP is crippled to force registration. ULP
is fully functional, and will always remain so. The primary difference is no
time delay and beg message once registered. The time delays will begin 45 days
after the initial installation of ULP.
Why register? Besides a clean conscience, you will get a registration key that
will work for all future versions of ULP, and will remove the delay and message
at the end of execution of each program.
The registration fee for ULP is $25 for hobbyist BBSes. The registration fee
for commercial BBSes, defined as running your BBS in the course of a commercial
business (e.g. more than 10 nodes), is $40. Please print the file REGISTER.FRM
and fill it out. You can print out the form by issuing the following command
from the DOS prompt:
TYPE REGISTER.FRM > PRN
┌───────────────────────────────────────┐
│ 6. License, Warranty and Disclaimer │
└───────────────────────────────────────┘
I'll keep this part short and sweet, and dispense (mostly) with the legal-ese:
License: You are allowed to use ULP for 30 days, after which you must
either register ULP or stop using it completely. Decompiling,
disassembly or any other form of reverse engineering the ULP programs
for any purpose is prohibited. ULP registration is a license for your
use of ULP; I retain ownership of the software. A single registration
applies to a single BBS system, regardless of the number of computers
used in the system. If you run two or more distinct BBS systems on the
same computer or network (with different names), you require two or
more ULP registrations. ULP registrations are not transferrable; you
cannot sell your registration to another sysop. Refer to the
registration form for the currect pricing structure.
Warranty: There isn't one. The only thing I'll guarantee is that ULP will
take up disk space, and will disappear when deleted.
Disclaimer: I'm not responsible for anything bad that happens. ULP works
here, but I cannot be held responsible for it not working on your
computer or doing any damage to hardware or software.
If these aren't agreeable with you, then the best thing to do is delete ULP
right now. I'll do my best to help any user (registered or not) that wants to
use ULP, and I'll act on bug reports as quickly as possible, but I simply
cannot and will not be responsible for anything bad, like lost data, disk
crashes, or whatever else you can think of.
┌────────────────────────────┐
│ 7. Conceptual Background │
└────────────────────────────┘
*******************************************************************************
READ THIS SECTION CAREFULLY, AS IT WILL MAKE LIFE MUCH EASIER!!!
*******************************************************************************
Since the ULP system is much more comprehensive than other upload processing
software, and therefore more complex, this overview and concept explanation
should help you understand how ULP is designed and how it can be best used for
your BBS. Some of this information may be specific to PCBoard sysops, but the
general concepts remain the same.
Note that ULP is an inter-operating set of tools, not distinct independent
functions. In this fashion, ULP is better able to track what is happening
between online testing and the event without sysop intervention.
BATCH PROCESSING:
─────────────────
As a minimum setup, you MUST run ULP as an event-mode batch processor, since
ULP handles most of the database updating, archive conversion, file and
description moving, archive information line computation, and other features
during the batch run. THIS IS NOT AN OPTION!!! Even if you run everything
online in slow mode, some quick maintenance needs to be performed by ULP in the
event on ULP system files that are not safely manipulated online.
When running ULP in a batch mode (batch, override), such as processing new
files that come in on tape, CD-ROM, file distribution network, etc., ULP's
operation can summed up as follows:
ULP looks in the source area and processes all new files it finds (files it
hasn't seen before). ULP moves the good files to the destination area, if
one is defined. ULP moves the defective files to the failure area, if
defined.
Pretty simple concept, huh? <g> ULP's batch modes do what you tell it to in
the upload directory areas configuration; it doesn't know or care about private
areas, public areas or otherwise. Through the use of a self-maintaining data
file, ULP knows what it has and hasn't already processed in every source
subdirectory configured. This allows enormous flexibility for a variety of
tasks. Some possibilities:
- Local uploads: Configure an input directory as your ULP source area, and
your BBS new uploads directory as the output. As you get new files,
such as downloads from your own BBSing activities, place any files you
want to locally upload into the input directory and the batch
processing will process all new files and move the good files into your
BBS new uploads directory. Optionally, a failure directory can be
configured to have the bad files moved into for better organization.
- Robocomm R&P: This would be setup similarly to the local uploads, where
you pass the Robocomm download directory and download DIR listing that
it creates during R&P runs to ULP as the source area, and configure
your new uploads area as your ULP destination area.
- File distribution nets (.TIC files): ULP doesn't toss .TIC files (no sense
re-inventing the wheel), but ULP's batch processing works very well
with most any .TIC tosser. There are two ways this can be implemented.
First, the .TIC tosser can toss all the files into a holding directory,
and then ULP can move the good ones to the new uploads subdirectory.
If this way is used, you simply need to configure the holding directory
as the ULP source area, and the new uploads directory as the
destination.
The other way is to have the .TIC tosser toss the files to their final
destination in your file directories. The requires that you configure
every subdirectory that the .TIC tosser may toss files into as the
source directory, leaving the destination area blank. This allows ULP
to scan all files in all directories, selecting only the new ones, and
process them. In this implementation, it would be a good idea to
configure a failure area so that defective files are moved out of the
available file area to prevent user access to them.
ULP's batch processing modes are designed to be able to be run without shutting
down your BBS nodes, however it would be a good idea to disable uploads during
batch processing. The reason for this is that if an upload occurs in a
directory currently under event processing, the directory listings may become a
source of contention if they both try to update it simultaneously. This is
very small risk, but it was documented anyway just in case.
ONLINE TESTING:
───────────────
I believe that all responsible BBS sysops verify all of their uploads prior to
posting them, in order to protect both themselves and their users. ULP is
designed with idea in mind. Most, if not all, sysops process uploads in one of
two ways (listed with benefits and liabilities as I see them), regardless of
what upload processing software they use:
1) Make all uploads private, processing them during a system event for
public distribution.
BENEFITS:
∙ Takes up very little on-line time on the user's part to process
archives if the re-compression process is skipped while online.
∙ Allows the conversion of all archives to a default format, so that
the BBS archives are consistent.
∙ Allows the BBS to accept any archive format...face it, it's hard
enough to get some of these weenies to upload, much less compress
them the same way.
LIABILITIES:
∙ Files are not available immediately for download.
∙ Does not catch duplicates or aged archives until after the user
has uploaded them, and perhaps leads to abuse by clever (?) users.
(It is assumed that these sysops still use the venerable 'PKUNZIP
-T' in their PCBTEST.BAT...)
2) Process (test) each upload online after the user uploads them, and
making them available for immediate download.
BENEFITS:
∙ Catches duplicate, defective and aged archives while the user is
online, denying him upload credits.
∙ Files are available immediately for download if they are not made
private in the PCBoard setup.
LIABILITIES:
∙ Takes up on-line time for a user, potentially adding to his
long-distance phone bill, discouraging further uploading; this
process is typically quite slow for large archives.
ULP also implements an online processing system, with varying modes of
operation from complete processing to a very fast scan of the archive directly
for needed data. These modes will allow you to run your BBS in the most
efficient manner possible for both you and your users.
Pay attention to this part!!!:
PCBoard (as do many BBS platforms) normally has two upload directories for each
conference: a private and a public directory. When PCBoard invokes
PCBTEST.BAT, the upload is currently in the private directory. If the archive
fails the testing, it will remain in the source directory. However, if it
passes, one of two things will occur depending upon your system setup:
- If you have made all uploads private in PCBSETUP, the file will remain in
the private directory.
- If you have not made uploads private, it will be moved BY PCBOARD (not ULP)
to the public directory.
This is an important point: during online testing, ULP does not move the file
in any way; the BBS software is expected to disposition the upload file itself.
ULP feeds back the testing results to the BBS software in two ways, via
semaphore files and via an errorlevel. The errorlevels returned by ULP are
listed in the appendix of this file.
In PCBoard, if you have made all uploads private (via PCBSETUP), the setup and
configuration of ULP is a snap: the source directory is the private upload
directory, and the destination is the public directory. However, if you want
to allow users access to untested uploads, then your source directory is the
public upload directory, and the destination information is left blank. To
illustrate the operation:
MAKE ALL UPLOADS PRIVATE │ ALL UPLOADS AVAILABLE AFTER TESTING
───────────────────────────────────┼─────────────────────────────────────
2 directories: C:\PRIVATE │ Same...
C:\PUBLIC │
│
User uploads a file, gets placed │ Same...
in C:\PRIVATE by PCBoard │
│
ULP tests it online │ Same...
│
PCBoard leaves file in C:\PRIVATE │ If it passes, PCBoard moves it to
│ C:\PUBLIC; if it fails, PCBoard
│ leaves it in C:\PRIVATE
│
ULP in the event processes all │ ULP in the event processes all *new*
new uploads found in C:\PRIVATE │ uploads found in C:\PUBLIC since the
since the last event and moves │ last event
all good uploads to C:\PUBLIC │
Further, ULP's online testing modes also have three different modes of
operation: slow, normal and fast. Slow mode completely tests the upload
exactly as ULP does in the batch event. Note that you MUST use the "all
uploads public" mode of operation to run slow mode (this shouldn't be a
problem, since there's little logic in forcing a user to wait for complete
processing of an upload, only to be held private until later).
ULP's online normal mode de-compresses the files, performs file, duplication
and age checking, and then deletes the extracted files and returns to the BBS,
informing the BBS of the test results. It does not re-compress the archive,
remove BBS ads, add information lines, etc.; this is saved for the event
processing. This mode can be used with both setup paradigms, making all
uploads private or public.
Fast mode DOES NOT decompress the file; it firsts performs an archive integrity
check, scans ARJ and ZIP archives directly for duplicate and age information,
and then returns to PCBoard (if the archive is not ARJ or ZIP, then normal mode
is forced). Typical fast mode scans of multi-megabyte archives are performed
in less than 5 seconds! In fast mode, file checking (viruses, etc.) is left
for ULP to do in the event (which is why the above discussion regarding
private/public directories is important). This mode can also be used either in
making uploads public or private, although it would be a good idea to make them
private with this mode, since the uploads are not file-checked (e.g. for
viruses) during test.
ULP's online testing modes will also accept a redirected ARJ or PKUNZIP listing
text file with the special name VERIFY.ULP (no other name is acceptable) as
input to pre-verify an upload for a user, before the user actually spends his
time uploading the file only to find out it won't pass the limits you set.
┌───────────────────┐
│ 8. Installation │
└───────────────────┘
Now, on to installation of the ULP system. ULPSM makes the installation ULP
very easy, since it has a built-in initial installation system and context-
sensitive, cross-referenced and fully indexed online help. After reading the
general concepts described in the preceding section, the following outlines a
general guide to installing the ULP system in the shortest time:
1. You will need the following programs and utilities for the default ULP
installation:
ARJ, LHA and PKZIP/PKUNZIP archivers (DOS installations)
LH/2 and ZIP/UNZIP archivers (OS/2 installations)
TXT2MSG message importer (both DOS and OS/2 installations)
Many other utilities can be used in conjunction with ULP, but these are
assumed to be available in the default ULP configuration.
2. Make a subdirectory on your hard drive. For the purposes of this document,
we'll call it "C:\ULP". Unarchive the ULP distribution archive into this
subdirectory. You've more than likely already made it this far, since you
are reading this file. <g>
* NOTE: The help file ULPSM.HLP is required by the ULP programs and must be
located in the same subdirectory as the ULP executables for proper
operation!
3. The ULP system opens several files at once for various reasons. You should
have a minimum of FILES=40 per node in your system CONFIG.SYS file, since
ULP is run in conjunction with your BBS. On the machine/node that runs the
ULPSM duplication database compilation event, I suggest at least FILES=50,
since the ULPSM database compilation routine opens up to 20 files
simultaneously.
4. If you are running ULP under a network or a DOS multitasking operating
system, you should already have DOS's SHARE.EXE loaded (unless you are using
a SHARE-compatible network operating system such as Novell). You must have
SHARE capabilty loaded in order to take advantage of the file sharing and
locking methods used by the ULP programs to prevent data loss. (If you are
running a single-node system without a multitasker, SHARE is not needed).
* NOTE: MS-DOS has a documented bug when SHARE is loaded high, where it
loses the table in memory. Load SHARE low to prevent potential
sharing problems!
5. Make sure you have your BBS software set to swap out of memory when running
an upload processor. This is required due to memory requirements of
archivers, virus testers, etc., and probably is already set for most BBS
packages. Check anyway... <g>
6. Run ULPSM using the following command line:
ULPSM -CULP.CFG
The first time you use a new configuration filename in the ULPSM command
line, ULPSM will create a configuration file and generate the required
external files that a standard ULP implementation uses. When you are asked
to save the configuration, save it. From this starting configuration, only
a few specific adjustments need to be made. Note that the pre-configured
archiver command lines are different based upon whether the DOS or OS/2
version of ULPSM is used to create the configuration file.
7. Run ULPSM again, with the same command line as above. You will need to make
the following configuration edits to complete your installation; don't
forget about the online help...there's a lot of useful information in there!
In addition, ULPSM will check for common configuration errors and provide
messages indicating any problems that it may find.
Menu A (General options):
- If you are running PCBoard, enter the full path and filename for
your DOWNLOAD.TXT file if automatic updating is desired. Otherwise,
leave it blank.
Menu B (Upload directories):
- Select an unused slot and press ENTER. Enter the appropriate
subdirectories and directory listings for your upload processing.
Refer to the following section titled "Configuration" and the online
help for more detail on this topic. For the time being, you may
only want to configure one or two sets of upload directories for
testing purposes.
Menu E (Archivers):
- ULPSM pre-configures the most common archivers used (ARJ, LZH and
ZIP for DOS, and LZH and ZIP for OS/2). If one or more of these
archivers are not available in your DOS or OS/2 path, you will need
to either put them in the path, provide the path directly in the
command lines in ULPSM or remove them from the ULP configuration.
Putting the required archivers in your path is more than likely the
simplest solution.
Menu F (Virus/file testers):
- Due to the wide variety and utilization of virus scanners available,
ULPSM makes no assumptions as to what you wish to use. In this
menu, configure one or more file and virus testers to be used by
ULP. Pay special note to the ULPSM help system, as it contains
recommended command lines for many of the more popular DOS and OS/2
virus scanners.
Menu G (Duplication checking):
- If you are using the ZDCS duplication system, you will need to
change the default setting from 'I' to 'Z', and provide the path to
ZDCS in the "Edit duplicate checking parameters" submenu.
- If you are using the internal duplication system, you will need to
build and compile a database. Select submenu B (Add files to
duplication database) and use one of your download directories as a
starting point. After ULPSM is finished, select submenu C (Compile
duplication database) to compile the database. More detail on
building your database will follow; for now, this will get us a
working database to tinker with.
Menu J (GIF/JPEG file testing):
- If you wish to use a graphics checking program (e.g. GFXCheck or
GIFTEST) to test GIF and/or JPEG uploads, place the command line in
this menu. Again, refer to the online help for recommended command
lines.
Menu L (Online testing):
- ULPSM defaults to the normal online testing mode with a switchover
to fast mode at 500K. If either slow or fast testing modes are
desired, alter the settings.
Menu M (Process data file):
- Select submenu B to initialize the process data file. Generally,
initialization is required only once, unless otherwise instructed by
ULPSM.
Menu N (Advanced options):
- If you are not running PCBoard as your BBS software, these settings
may make integration of ULP into your system easier. If you are
running PCBoard, adjustment of these values is usually not necessary
and will probably break your setup.
Exit ULPSM, saving your configuration edits.
8. Take a look at your ULP subdirectory now. You will see a dozen or so
external configuration files that ULPSM automatically created. These can be
edited either with a conventional text editor or through ULPSM using the
F2/F3 hotkeys, where appropriate.
One of the newly-created files needs to be relocated. Copy the PCBTEST.???
file to your \PCB subdirectory, copying over the existing one. Back up your
old one if you think you might want it back later (I don't think you will,
though <g>).
You will also find a file named ULPBLT. This is a generic bulletin that you
may wish to post on your BBS to explain to your users the process for pre-
verifying uploads to your system via ULP.
9. You are now all configured, and almost ready to go. You now need to build
a complete ULP database if you are using the internal duplication database.
This can be postponed until later if you wish to do some testing or
experimentation with ULP prior to investing the time in constructing a
complete duplication database. Refer to the section titled "Internal
Duplication Checking" for additional discussion.
10. Edit your system event file, adding the following lines somewhere in it:
C: ─────────────────┐ Change as necessary
CD \ULP ─────────────────┘ to match your setup
ULP -CULP.CFG -MBATCH -T0
ULPSM -CULP.CFG -S -T0
This will run the ULP event batch processor and then compile your
duplication database with ULPSM (the last line is required only if you are
using ULP's internal duplication database system).
11. That wasn't so bad, now was it? Feel free to poke around with other
settings once you have verified that ULP is functioning properly.
┌────────────────────┐
│ 9. Configuration │
└────────────────────┘
Most of the information required to install ULP is contained in the online help
system in ULPSM. This exhaustive information will not be repeated here for
brevity, but some topics that require additional clarification will be
addressed here.
UPLOAD DIRECTORIES:
───────────────────
In the past, this topic has been a point of some confusion to sysops. ULP
makes use of the upload directories configured in ULPSM during both online and
batch processing, but in different ways.
When testing a file online, ULP is provided all necessary information by the
BBS software. It really doesn't need any of the upload directories defined in
the ULP configuration; however, it does a comparison to the path passed by the
BBS software and the areas configured to try and find a match. If a match is
found, ULP will use any area-specific settings (e.g. duplication and age
limits) that are configured. If no match is found, ULP will spit out a warning
and continue with the default settings.
On the other hand, during batch (and override) processing, ULP makes extensive
use of the upload directory configuration. It will go through all upload
directory sets configured, in order, processing any new files found in the
source area, moving the good files to the destination (if configured) and any
defective files to the failure area (if configured).
It is extremely important that you do not use one upload directory set's
destination or failure subdirectory as a another set's source directory. This
can cause files to processed multiple times, resulting in duplication failures
and other headaches.
Generally, if you make all uploads private, your source area will be your
private upload area and the destination area will be your public upload area.
If you make all uploads public, the source area will be your public upload area
and the destination area will be blank. In either case, failure areas can be
used if desired. Refer to the section titled "Conceptual Background" for more
detail.
ONLINE TESTING:
───────────────
To use ULP for on-line testing of archives, the following command line should
be invoked. During the initial configuration, ULPSM creates a PCBTEST.BAT for
PCBoard sysops to use, which should be all that is necessary for PCBoard
sysops.
C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -T0
However, for non-PCBoard sysops, the following discussion outlines how to
install ULP for online testing in your BBS package.
Two of the three passed % parameters to the batch file are required: the
filename to test, passed as %1 by PCBoard and the mode, passed as %2. The
temporary description file, passed as %3, is not required but will be updated
by ULP if passed.
The filename (-F switch) should include a complete pathspec. ULP will compare
the path to the source upload directories configured in ULPSM, and if a match
is found, ULP will use any directory-specific settings configured. Therefore,
the path(s) configured in ULPSM should match those configured in your BBS
software for new uploads.
The only acceptable modes (-M switch) for online testing are "UPLOAD", "TEST"
and "ATTACH" (these correspond to PCBoard's modes). Upload mode is basically a
full process, and is the most commonly-used mode. Test mode unpacks the
upload, performs file-checking and returns; this is done to allow the online
user to test a file on the BBS in case a problem is found after download.
Attach mode performs a complete test, but doesn't fail the upload under any
circumstances; this is used primarily for attaching file to messages on the
BBS.
The temporary description file (-D) is created by the BBS software and is
expected to be in a format identical to the other upload directories as
configured. If it is not, do not use this parameter and let the event mode
execution of ULP update the description.
SERIAL I/O DEFINITION:
──────────────────────
ULP will gather any information that is required for operation from the door
drop file located in the startup subdirectory (PCBOARD.SYS and DOOR.SYS are
supported). If desired, the serial port information can be handed directly to
ULP by using the -P and -B command switches:
C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -P3E8,5 -B57600
The -P parameter can be 1 or 2, representing standard COM1 and COM2, or in the
format "address,IRQ" as illustrated above for non-standard serial ports. If -P
is set to 0, this will suppress the remote comm I/O entirely. The locked DTE
baud rate of the port (not the DCE carrier speed) can be passed via the -B
parameter.
If you are using the DSZPORT environment variable to define the port IRQ and
address to DSZ, ULP can also use this information as well. If the drop file
indicates that a non-standard port is in use (e.g. not COM1 or COM2), ULP will
automatically look for the DSZPORT environment variable. Refer to the DSZ
documentation for the proper specification of the DSZPORT environment variable
(short version: it looks like the -P parameter, "SET DSZPORT=3E8,5").
ULP is capable of using a FOSSIL driver, but you must tell it to do so via the
ULP command line. Use the -X command switch to tell ULP to use the FOSSIL
driver:
C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -X
ULP will use the port specified in the door drop file or on the command line
via the -P switch. As a quick technical aside, with most FOSSIL drivers, the
FOSSIL port number is normally 1 less than the COM port number. In other
words, COM1 is FOSSIL port 0, COM2 is FOSSIL port 1, etc.. ULP handles this
conversion internally, but if necessary, you can force a specific port number
via the -P command line parameter:
C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -P1 -X
The above command line will force ULP to use FOSSIL port 0, which is by
convention, COM1.
Similarly, the DOS ULP is also capable of talking directly to the OS/2 SIO API.
In order to use this mode, you must tell ULP to do so via the -O command line
switch:
C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -O
Note that the OS/2 version of ULP obviously must use the OS/2 SIO API, so the
-O switch is not required (but is accepted for compatibility's sake).
REMOTE DISPLAY TEMPLATES:
─────────────────────────
The remote display templates offer tremendous flexibility for sysops to
customize the user display that is produced by ULP. While the template file
design may seem a bit odd, it provides maximum flexibility for customization.
Basically, display templates are comprised of two sections: the form, and the
responses. When ULP is started up to perform an online test, the appropriate
template is loaded and the form is immediately displayed. Special {-variables
are used in the form to tell ULP where in the form to put the responses.
As the file is processed by ULP, at special points in the processing ULP will
output the responses, selecting the appropriate response based upon pass or
failure. If a process is not performed (such as packing when testing in normal
or fast modes), no response is output for that process. The form can be edited
to eliminate these unnecessary lines if desired and appropriate.
As an example, review the following simple display template:
;
; ****************************************************
; * UpLoadProcessor sample remote display template *
; ****************************************************
;
; Form definition
;
BEGIN_FORM
Verifying @FILENAME@ from @USER@ on node @NODE@...
┌─────────────────────────┐
│ UpLoadProcessor @VERSION@ │ Registered to: @BOARDNAME@
│ (c) 1992-96 Stacy Smith │ [Processing at @SYSTIME@ on @SYSDATE@]
└─────────────────────────┘
Upload format: {FMT}
Unpacking file: {UNP}
Testing file integrity: {CHK}
Duplicate checking: {DUP}
Age checking: {AGE}
Packing archive: {PCK}
Other checks: {MSC}
END_FORM
;
; Response definitions
;
BEGIN_RESPONSES
@OPTEXT@
@OPTEXT@
Unpacked OK
Unpacking failure!
Passed virus checks
Failed virus checks
Passed duplication
Failed duplication
Passed age
Failed age
Compressed OK
Compressing failure
OK
Failed!
END_RESPONSES
;
; End of UpLoadProcessor template file
;
The actual form is sandwiched between the two keywords "BEGIN_FORM" and
"END_FORM"; 18 lines by 78 columns is the maximum size allowed in the form.
Various @-variables can be used in the form and responses. A complete list is
in the ULPSM online help; unsupported @-variables will not be translated.
There is no support for @CLS@; the reason for this is that ULP automatically
clears the screen before displaying the form. This allows ULP to properly
determine the screen location of responses on a consistent basis.
Note the special variables {FMT}, {UNP}, {CHK}, {DUP}, {AGE}, {PCK} and {MSC}.
These variables do nothing except tell ULP where the specific responses are to
be located, and have a different format to help prevent confusion. During form
display, they are removed with no output so that they can ignored when
attempting to align text. The following table defines the meaning of these
seven special macros:
{FMT} Location for the upload file format known/unknown response
{UNP} Location for the unpacking pass/fail response
{CHK} Location for the file/virus checking pass/fail response
{DUP} Location for the duplicate checking pass/fail response
{AGE} Location for the age checking pass/fail response
{PCK} Location for the packing pass/fail response
{MSC} Location for the miscellaneous pass/fail response
The "miscellaneous tests" defined are primarily the TCANs. This response is
always the last response displayed during processing.
Similarly, the responses are sandwiched between the two keywords
"BEGIN_RESPONSES" and "END_RESPONSES". @-variables can also be used in the
form and responses, but all responses must be 50 characters or less. The order
of the responses is critical; no responses can be skipped (but they can be left
blank). The response order is as follows:
BEGIN_RESPONSES
Archive format identified response string (usually just @OPTEXT@)
Archive format unknown response string (usually just @OPTEXT@)
Unpacking passed response string
Unpacking failed response string
File/virus checking passed response string
File/virus checking failed response string
Duplicate checking passed response string
Duplicate checking failed response string
Age checking passed response string
Age checking failed response string
Packing passed response string
Packing failed response string
Miscellaneous tests passed response string
Miscellaneous tests failed response string
END_RESPONSES
A special macro called @OPTEXT@ can be used in the responses. For each
possible response, the @OPTEXT@ macro will be loaded with special process-
specific information, such as duplication ratio and archive age. The following
list describes the contents of the @OPTEXT@ macro for each of the seven
possible responses:
{FMT} @OPTEXT@ contains the archiving format extension or "Unknown"; if
the archive has an -AV or is secured, @OPTEXT@ will be appended
with "(-AV/secure)".
{UNP} @OPTEXT@ contains the program name used to unpack the upload.
{CHK} @OPTEXT@ contains the last file checker executed (primarily for
failures).
{DUP} @OPTEXT@ contains the total and executable duplication ratios in
the format "100%/100%".
{AGE} @OPTEXT@ contains the age of the archive in months.
{PCK} @OPTEXT@ contains the program name used to pack the upload.
{MSC} @OPTEXT@ contains the name of the test (primarily for failures).
Note that by displaying results to the remote user can be a double-edged sword.
While it can prevent the requisite questions from a user as to why an upload
failed, this could also allow the "clever" user to bypass a failure by tweaking
the archive contents just enough to make it pass. Consider this before you use
the @OPTEXT@ response macros.
All comments in the template file begin with a semi-colon, however, comments
are not permitted between the BEGIN_FORM and END_FORM form definition or
between the BEGIN_RESPONSES and END_RESPONSES response definition.
* NOTE: While the remote display template system is designed primarily for
ASCII and ANSI, RIP screens *may* be possible (I don't know as I
haven't tried it) by creating the appropriate RIP script command
strings in the form and response definitions. However, the RIP
graphics will not be shown in the local display window.
For those who may be curious, the above example template will produce a display
that looks like (for a passed upload):
Verifying FOO.ZIP from JOE BLOW on node 4...
┌─────────────────────────┐
│ UpLoadProcessor 2.20 │ Registered to: Nice Guys 'R Us
│ (c) 1992-96 Stacy Smith │ [Processing at 12:01 on 01/02/95]
└─────────────────────────┘
Upload format: ZIP
Unpacking file: Unpacked OK
Testing file integrity: Passed virus checks
Duplicate checking: Passed duplication
Age checking: Passed age
Packing archive: Compressed OK
Other checks: OK
By using the F4 hotkey in ULPSM, you can preview your display template, taking
into consideration your current ULP configuration.
ULP fully supports multi-lingual display templates through the use of PCBoard
conventions. If you are offering multiple language PCBTEXT files, you can
create multiple language display templates of the same extension and ULP will
automatically select the appropriate language template, if one exists (if not,
ULP will use the base template).
COMMENT TEMPLATE:
─────────────────
The comment template allows customization of a comment for each file processed
through the use of @-variables. Archive statistics, file descriptions,
processing date and time, BBS advertising, etc. can all be implemented in this
template for display by using @-variables. A complete list of @-variables and
macros is available in the online help. The following is a suggested comment
template:
┌─────────────────────────────────────────────────────────────────────────┐
│ This archive has been fully tested and verified using UpLoadProcessor │
│ by your Sysop to ensure your system's safety and file quality! │
└─────────────────────────────────────────────────────────────────────────┘
Processed at @SYSTIME@ on @SYSDATE@
Archive statistics:
Number of files: @NUM@
Oldest file: @OLD@
Newest file: @NEW@
Total bytes: @BYTE@
Archive description:
───────────────────────────────────────────────────────────────────────────
@DESC@
───────────────────────────────────────────────────────────────────────────
MESSAGE TEMPLATES:
──────────────────
The message templates also offer a high degree of flexibility for sysops to
customize messages automatically generated by ULP during upload testing to the
online user and/or the BBS sysop. By default, a user message is sent to the
online user and the sysop message is sent to "SYSOP" in conference number 0
(the main board conference for PCBoard systems) using a third-party message
insertion utility such as TXT2MSG for PCBoard.
The message template file(s) include both a fail message and a pass message;
this allows for different messages to be sent based upon test results. Two
special control statements can be used to alter both the recipient and
conference (MSG_TO_NAME and MSG_CONF, respectively). Review the following
sample user message template:
;
; *******************************************
; * UpLoadProcessor user message template *
; *******************************************
;
MSG_FROM_NAME UpLoadProcessor
;
; Failed upload message
;
MSG_CONF 1
MSG_OPT -rly
BEGIN_FAIL_MSG
Hello @FIRST@,
Thanks for the upload of @FILENAME@.@OFEXT@; unfortunately, it failed
UpLoadProcessor's validation for the following reason:
@FAILTEXT@
If you believe this disposition was made in error, please contact the
sysop.
Thanks again for uploading and please try again soon!
END_FAIL_MSG
;
; Successful (passed) upload message
;
MSG_TO_NAME all
MSG_CONF 0
MSG_OPT -ply
BEGIN_PASS_MSG
Hello all,
@USER@ has uploaded the new file @FILENAME@.@NFEXT@ to conference
number @CONFNUM@:
Upload size: @NSIZE@ bytes
Number of files: @NUM@
Dated: @OLD@ - @NEW@
Description follows
---------------------------------------------------------------------
@DESC@
---------------------------------------------------------------------
If this file looks of interest to you, take a look in the new uploads
directory for it. Thanks to @FIRST@ for the upload!
END_PASS_MSG
;
; End of UpLoadProcessor template file
;
Notice that the message bodies are sandwiched between two pairs of control
statements, "BEGIN_FAIL_MSG" / "END_FAIL_MSG" and "BEGIN_PASS_MSG" /
"END_PASS_MSG". If no fail or no pass message is desired, then that message
section of the template can be omitted. All comments in the message template
file begin with a semi-colon, however, comments are not permitted in the
message body.
Most of the available @-variables can be used in the message body; a complete
list is in the ULPSM online help and unsupported @-variables are not
translated. Note that @X color codes should not be used, as they will be
translated into ANSI codes which is not desirable for message bases.
In the above example template, the fail message has been redirected to
conference number 1 via the "MSG_CONF 1" before the fail message body line, and
the pass message directed to conference 0 to all users via the "MSG_CONF 0" and
"MSG_TO_NAME all" lines before the pass message body. Furthermore, through the
user of the "MSG_OPT" parameter, the fail message has been made private (-r for
TXT2MSG) and the pass message is public (-p). For the above example template,
the fail message will look something like:
Hello Joe,
Thanks for the upload of FILENAME.ZIP; unfortunately, it failed
UpLoadProcessor's validation for the following reason:
Age (60 months)
If you believe this disposition was made in error, please contact the
sysop.
Thanks again for uploading and please try again soon!
and the pass message:
Hello all,
JOE BLOW has uploaded the new file FILENAME.ZIP to conference
number 0:
Upload size: 123456 bytes
Number of files: 10
Dated: 01-01-95 - 01-01-96
Description follows
---------------------------------------------------------------------
FILENAME.ZIP 123456 01-01-96 Cool brand spaking new file!
---------------------------------------------------------------------
If this file looks of interest to you, take a look in the new uploads
directory for it. Thanks to Joe for the upload!
Failure-specific message templates can also be created by appending the failure
extension to the base filename. For example, if your generic user template is
named "USER", a special template for duplication failures (extenion ".DUP") can
be used by naming it "USERDUP". This can be of great help for sysops who get a
lot of questions regarding certain types of failures. A complete list of
failure extensions is located elsewhere in this document. No pass message
definition is required for failure-specific templates; the pass message will
always be extracted from the base (generic) message template.
ULP also supports multi-lingual message templates. If you are offering
multiple language PCBTEXT files, you can create multiple language message
templates of the same extension and ULP will automatically select the
appropriate language template, if one exists (if not, ULP will use the base
template).
┌─────────────────────────────────────┐
│ 10. Internal Duplication Checking │
└─────────────────────────────────────┘
When first installing ULP, if you plan to use ULP's internal duplication
database system, the duplication database must be created from scratch. If you
have mostly ZIP and ARJ files, then this can be very quick (on the order of 5
minutes per 1000 archives on a hard disk, CD-ROMs typically take between 30
minutes and an hour...these numbers are from my experience, your mileage may
vary). As usual, the online help provides a lot of useful information on
specific duplication database manipulation options.
If you use CD-ROMs on your BBS, pre-built databases may already be available
for your CD-ROM, greatly reducing the amount of time required to get your
system ready. ULPSM can merge these pre-built duplication databases with your
main database in a matter of seconds versus spending the time building it
yourself.
If you are a user of the FWKCS duplication database system, ULPSM can import
and translate the FWKCS database directly into the ULP format. This will also
be preferable from a speed standpoint to rebuilding the database from scratch.
If you have files that are off-line, they can be added to your duplication
database fairly easily. Simply copy your offline files to a temp directory
(for the sake of argumentation, let's call it "C:\TEMP"). You can then add
them to the duplication database via ULPSM. After the offline files are added,
just delete them from the temporary subdirectory, and if someone uploads a file
that you already have off-line, it will be rejected by ULP.
Once you have your database built, regular maintenance on the duplication
database files is required. This will compile any new data from the day's
uploads into the main database, and remove any added temporary data from ULP's
online testing. This is not required to be done every day, but performance
will degrade as more and more files (e.g. hundreds or thousands of uploads) are
processed without compilation.
To control duplication database size, database entries can also be purged from
the database. Removal is based upon the file date represented by the entry,
NOT the date the file entry was made into the database; these are not one and
the same. In general, this function is not required...ULP's internal database
system sees no performance degradation until the database exceeds 30 megabytes
(as a baseline, about 75 full CD-ROMs), which no one has even come close to
yet. If you do wish to purge, purge using an age of at least twice your
configured age limit...this will help ensure that required data is maintained.
Note that ULPSM also locks the duplication database, preventing any other
program, including ULP, from accessing it. I strongly suggest you have uploads
disabled when running ULPSM to compile your database to prevent upload failures
due to the inability to access the database. At the minimum, perform the
compilation at a time when uploads are not likely to occur.
As with any other database system, you should backup your ULP duplication
database and index files regularly. ULPSM validates your duplication database
during compilation to ensure that it hasn't been corrupted, but if corruption
occurs (due to crashes, power loss, deletion, bad karma, etc.), the database
may not be repairable. Therefore, backup regularly!!!
┌────────────────────┐
│ 11. Other Topics │
└────────────────────┘
While the ULP system is mostly automatic, there are some occasions where
operations may have to be done manually. The following topics discuss some of
the issues related to running ULP manually, including some command line
switches.
OS/2 SPECIFIC VERSION (ULP & ULPSM):
────────────────────────────────────
The OS/2 executables of ULP and ULPSM are native 32-bit executables that
require OS/2 version 2.0 or later. They are completely compatible with the DOS
versions of ULP and ULPSM, including all configuration and data files, but do
not have some of the functional limits imposed on the DOS versions due to
memory constraints.
However, it should be noted that separate configurations will be required if
you wish to use both the DOS and OS/2 versions of the ULP programs since the
external utilities are different in both name and command lines. For example,
since PKWARE hasn't updated their PKZIP program for OS/2 beyond version 1.02,
you will need to use an alternate ZIP program, such as InfoZIP's ZIP 2.0.1 or
later. These command lines are, of course, different, hence different ULP
configuration files are required.
In addition, during testing, enhanced speed has been noted when *not* running
external programs in a window. This is partly due to the VIO speed of OS/2 and
CPU overhead of the I/O redirection thread.
You should not use DOS external programs in the OS/2 version of ULP. However,
if you must use an occasional DOS program, you should use PipeDOS, a utility by
Scott Maxwell that correctly passes the program errorlevel back to the calling
OS/2 program, which is a requirement for ULP to operate correctly. After
installing PipeDOS per it's instructions, simply configure the DOS command line
as you normally would in ULPSM, but the first argument before the program name
will be "pipedos", e.g.:
pipedos command arg1 arg2 ...
PipeDOS-called DOS programs should not be run in a window. Also, PipeDOS-
called programs are very slow, so I recommend you use them only for programs
that are called infrequently and that don't have OS/2 counterparts (e.g. ARJ).
MICROSOFT WINDOWS (ULP & ULPSM):
────────────────────────────────
Do not enable idle detection in Windows 3.x or Windows 95 (e.g. don't check the
"Detect Idle" box of the Advanced Settings of the PIF file). Doing so may
cause Windows to take away so many time slices from ULP in the background that
it will appear to hang (until brought back into the foreground or a key is
pressed). ULP gives away time slices without requiring Windows to do it's
thinking for it (what little thinking Windoze does <g>).
DESQVIEW (ULP & ULPSM):
───────────────────────
The DOS versions ULP and ULPSM write text directly to video for speed. If you
experience bleed-through of the ULP video in DESQview, you may need to enable
the flag for those windows that will run the ULP software to tell DESQview that
applications write directly to video (first page of settings, near the bottom).
That should eliminate the bleed-through.
ONLINE HELP (ULPSM):
────────────────────
The online help is fully indexed and cross-referenced with other related topics
to ease navigation through the database. If some fields seem to have
relatively little information, this generally means that another topic,
normally on a higher-level menu, has more in-depth information. If you have
difficulty finding a topic, go into the index and locate the pertinent titles
to find the help you need.
If you do not have a mouse to help navigate ULPSM's online help screens, the
"Index" buttom can be activated by pressing the Alt-I key combination. Also,
the "Prev" button is accessed by Alt-F1 and "Quit" is the same as pressing ESC.
The general idea behind ULP's documentation is that this document outlines how
to best install and use ULP. The online help outlines detailed configuration
information and some instructional information on ULPSM.
DISPLAY READABILITY (ULP & ULPSM):
──────────────────────────────────
ULP and ULPSM both utilize time delays at certain points in the process to
allow users to see what's going on with ULP. The default time delay is 3
seconds, but can be altered between 0 and 10 seconds via the -T command line
switch. Generally, during the event batch processing and online processing of
uploads, you should set the time delay to 0 for maximum speed. For example, to
change the time delay to 5 seconds for a run of ULPSM:
ULPSM -CULP.CFG -T5
When manually running ULP or trying to debug a configuration error, it may be
helpful to use a longer time delay to see exactly what ULP is doing normally at
high speed.
QUIET (ULP & ULPSM):
────────────────────
ULP and ULPSM normally issue a beep on an error or warning, but the beep can be
disabled using the -Q switch:
ULP -CULP.CFG -MBATCH -Q
ULPSM -CULP.CFG -Q
ESCAPING FROM LONG PROCESSES (ULP & ULPSM):
───────────────────────────────────────────
When running long, time-consuming processes, namely ULP's batch processing
modes and ULPSM's database addition, it is possible to abort the process by
pressing the ESC key. Note that ULP and ULPSM may not immediately respond!
They must finish the current task before allowing you to abort the process.
Further, if you are processing multiple subdirectories in a run, the abort will
affect only the current subdirectory. ULP and ULPSM will start the next
subdirectory after cleaning up from the current abort. This allows you to pick
and choose what to process if desired.
GIF/JPEG PROCESSING (ULP):
──────────────────────────
By default, GIF and JPEG file detection and processing is enabled. In order to
disable all GIF and/or JPEG file processing, set all three of the minimum image
parameters to 0. In doing so, you disable not only the file processing, but
the file format detection, resulting in a GIF or JPEG upload being rejected as
an unknown format.
On the other hand, if you wish to accept *all* GIF or JPEG files, regardless of
image parameters, set the minimum image width and height to 0, but set the
minimum number of colors to 1 (all graphic images have at least 2 colors).
This will have the effect of detecting and processing the requested graphic
format without rejecting based upon image values.
DEBUGGING (ULP):
────────────────
When attempting to find a configuration error or track down a potential bug in
ULP, using debug mode will be a great help. Debug mode will slow down the ULP
display, force all external programs to run full-screen and dump virtually all
screen output to the log file. To use debug mode, add the -! switch:
ULP -CULP.CFG -MBATCH -!
ALL BUG REPORTS MUST BE ACCOMPANIED BY A DEBUG LOG for me to be able to figure
out the problem. Note that if you are experiencing a problem that causes a
system hang, the debug log is in the data work subdirectory with a filename
"$LOG????".
OVERRIDE MODE (ULP):
────────────────────
During the course of operation, ULP may (depending upon your configuration)
rename archives that have been found to be defective in some manner according
to the following convention:
.UNK Unknown archive format
.DOS DOS reserved keyword found in archive
.ERR Error occurred while unpacking archive (archive integrity failure)
.CHK Error found while file checking archive file (virus, etc.)
.DUP Excessive duplicate files contained in archive
.PCK Error occurred while repacking archive file
.AGE Age limit exceeded by archive file
.ENC Encrypted file found in archive
.TCN File was TCANned
.BAD Error found while testing GIF file
.RES Unacceptable GIF or JPEG resolution
.GLT GIF compressed with GIFLITE
.WRK Unable to create work space for processing file
.MTY Empty archive detected
If you feel that these files are acceptable after reviewing them, you can force
them to be accepted by using override mode, e.g.:
ULP -CULP.CFG -MOVERRIDE
This will re-process ALL files in the source area (or the failure area, if that
is where the defective files are) and ignore the results of TCANs, duplication
and age, moving them to the destination area (if configured). Override mode
will not override unpacking, packing, integrity and virus errors, however, for
the safety of your board and your users. Also, override mode is of little use
if your upload areas are not utilizing a destination area.
If selected files need to be overridden, but not the entire set of failed
uploads, you can specify a filespec on the command line via the -F switch. For
example:
ULP -CULP.CFG -MOVERRIDE -FFILENAME.ZIP
ULP -CULP.CFG -MOVERRIDE -F*.ARJ
Since override mode operates on the upload directory sets configured in ULPSM,
paths are not required or supported in the filespec passed in the -F switch for
override mode. Any included path in the -F parameter will be ignored by ULP.
RETEST MODE (ULP):
──────────────────
ULP can retest for viruses, etc. all archives found in the subdirectory passed
to it via the -F switch. ULP will not perform any TCANning, duplication
checking or age checking, nor will it repack the archive. It will simply
unpack and archive and file check it; this allows sysops to use newer versions
of virus scanners periodically to look for viruses that may have been missed
during the original file processing with an older revision of the virus
checker. Some example command lines:
ULP -CULP.CFG -MRETEST -FD:\PATH\
ULP -CULP.CFG -MRETEST -FD:\PATH\*.ZIP
CONVERT MODE (ULP):
───────────────────
ULP can also perform a mass conversion of the archives found in the path passed
to it to your default archiver. ULP will retest and convert all archives found
in the subdirectory indicated using the same processing flags as new uploads.
For example:
ULP -CULP.CFG -MCONVERT -FD:\PATH\
ULP -CULP.CFG -MCONVERT -FD:\PATH\*.ARJ
Convert mode does not perform any description insertion or updating; it is
primarily to permit mass conversion of archives in bulk from one format to
another.
LOCAL MODE (ULP):
─────────────────
When ULP performs online testing, it expects to find a door drop file and to
generate a remote display to a user over the modem. ULP also determines at
startup whether an upload is a local upload, however this requires a drop file
as well. By using the -L switch, you can force ULP to perform an online mode
test without a drop file:
ULP -CULP.CFG -MUPLOAD -L -FD:\PATH\FILENAME.EXT
This mode can be used to plug ULP into another program as a single file
processor where a remote display is not required or appropriate.
RETAIN ORIGINAL ARCHIVE DATE (ULP):
───────────────────────────────────
ULP normally stamps all uploads with the current date and time of processing.
If you wish to keep the original archive date, add the -R switch to the command
line:
ULP -CULP.CFG -MBATCH -R
Note, that this doesn't affect the date inserted in the directory listing for
the upload; this affects only the file date stamp on disk.
NODE NUMBER (ULP):
──────────────────
ULP normally gets the current node number from the door drop files, however
this can be overridden on the command line using the -N switch should the need
arise. Normally this is used only for online testing modes, since the ULP
program manages node contention on its own to prevent node number duplication
and processing collisions. An example of node number definition:
ULP -CULP.CFG -MBATCH -N100
Use this switch only if you absolutely need to; under most circumstances it is
not required. Note that node 0 is not an acceptable number.
DISABLE 16550 FIFO (ULP, DOS versions only):
────────────────────────────────────────────
If your BBS is running on an oddball 16550-compatible serial port, such as
early Western Digital UARTs and Hayes ESPs, you may get better performance by
disabling the FIFO operation of ULP during online testing mode. This is done
by adding the -1 switch to the command line in PCBTEST.BAT (online testing is
the only time where you might need to disable 16550 FIFOs):
C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -1
HANDSHAKE METHOD (ULP, DOS versions only):
──────────────────────────────────────────
Serial I/O requires that the ports handshake in some fashion to know when to
start and stop data transfer; two methods are commonly used, either separate or
in combination. The most common is hardware (also referred to as RTS/CTS) and
this is the ULP startup default. The other method, which is used primarily for
compatibility with older equipment, is software (also known as XON/XOFF). If
you need software or both types of handshaking in your system, you can specify
it on the command line:
C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -HSOFTWARE
C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -HBOTH
┌───────────────┐
│ 12. BBS Ads │
└───────────────┘
The ULP system includes a BBS ad removal feature based on CRC-32 calculation of
the file contents and other pertinent data. In this fashion, ULP can detect a
known ad file despite changes of the file name and date.
In order for sysops to be able to 'keep up' with new ads produced by the weenie
sysops who insert the @!&*#%$ things, ULPSM has two maintenance functions that
permit the sysop to update the BBS ads database with their own information.
First, ULPSM can scan a BBS ad file (or multiple files), and update the BBS ads
database with the required information. Don't worry about duplication within
the database, as part of the compiling process is to purge duplicate BBS ad
data. To add files to the BBS ads database, select menu G (BBS ad processing),
and select submenu B (Add new BBS ad to BBS ads database). The online help
contains all information that may be necessary.
Second, since the latest version of my master BBS ads database is always
included in the distribution archive, you don't want to keep changing your
database from release to release. Therefore, you can merge the new release
database with your existing BBS ads database via ULPSM as well. To merge the
distribution database with your own database, select menu G (BBS ad
processing), and select submenu C (Merge pre-built BBS ad database). Again,
the online help contains all information that may be necessary.
Note that ULPSM cannot (will not) put 0-byte bbs adds in its database; this is
because every 0-byte file has the same CRC-32: 0. Therefore, inserting 0-byte
files into a numerical database is useless and doing so would remove *every*
0-byte file encountered, not just the BBS ads, including subdirectory markers
in pathed archives. The exclusion filter list is the best place for 0-byte
ads, since the only thing unique to 0-byte files is the filename.
I would greatly appreciate your passing along any new BBS ad files that you may
collect over time to me so I can update the master database that I include with
the ULP distribution archive. Please refer to the top of this document for my
BBS number.
┌─────────────────────────┐
│ 13. The Future of ULP │
└─────────────────────────┘
ULP will be supported as long as I'm in the BBSing business (which will be
quite a while...once it's in your blood, you can never shake it <grin>). The
ULP system will be rapidly expanding it's features so it will be your first
choice in BBS upload processors. Some current plans (in no particular order):
∙ Add the ability to preprocess files prior to file checking them;
for example, decompress executables that have been PKLite'd.
∙ Better support for other BBS software directory list formats.
If you have any other suggestions, or want other archivers supported, please
contact me via Email, U.S. snail-mail or on my BBS at the number at the top of
this document.
Thanks for giving ULP a try!
┌────────────────────────────────┐
│ Appendix A: DOS Errorlevels │
└────────────────────────────────┘
The following is a partial list of errorlevels returned to the operating
system by the ULP and ULPSM programs. Not all errorlevels are documented,
since the error messages are much more useful in debugging problems.
0 Successful execution (ULP and ULPSM)
1 Successful execution, nothing to do (ULP event modes)
1 Unknown archive format (ULP online modes)
2 DOS reserved keyword found in archive (ULP online modes)
3 Error unpacking archive (archive integrity) (ULP online modes)
4 Error file checking archive files (virus, etc.) (ULP online modes)
5 Error duplicate checking archive files (ULP online modes)
6 Error packing archive (ULP online modes)
7 Age limit exceeded by archive files (ULP online modes)
8 Encrypted file found in archive files (ULP online modes)
9 TCANned file (ULP online modes)
10 Bad GIF file (ULP online modes)
11 Unacceptable GIF or JPEG resolution (ULP online modes)
12 GIF compressed with GIFLITE (ULP online modes)
13 Unable to create work space (ULP online modes)
14 Empty archive detected (ULP online modes)
99 Help screen (executing a program with no or an insufficient number
of arguments) (ULP and ULPSM)
>99 Program error (refer to the error message and log file).
┌────────────────────────────────┐
│ Appendix B: Acknowlegements │
└────────────────────────────────┘
The DOS versions of ULP and ULPSM use the SPAWNO swapping routines by Ralf
Brown to minimize memory use while shelling to DOS and running external
programs.
The Graphics Interchange Format(c) is the Copyright property of CompuServe
Incorporated. GIF(sm) is a Service Mark property of CompuServe Incorporated.
(Standard verbage required by CompuServe)
The problem of automatically recognizing duplicate files on large bulletin
board systems, independent of filename, was originally solved by Dr. Frederick
W. Kantor (founder of Information Mechanics and author of FWKCS(TM)
Contents_Signature System), who came up with the elegant solution of using both
the 32_bit CRC and the uncompressed file length together to make a
"contents_signature" for each file. Dr. Kantor's experimental data has shown a
typical pairwise error rate of less than one part in ten trillion using this
technology.
┌─────────────────────────────────────┐
│ Appendix C: Command Line Summary │
└─────────────────────────────────────┘
ULP.EXE command-line syntax:
ULP -Cd:\path\config.ext -Mmode [-Fd:\path\<file.ext>] [-Dd:\path\desc.ext]
[-P#<,#>] [-B#] [-H?] [-1] [-X] [-O] [-L] [-N#] [-G?] [-R] [-T#] [-Q] [-!]
-C filename of the ULP configuration file
-M processing mode [Batch/Override/Retest/Convert/Upload/Test/Attach]
-F filename/subdirectory of the archive(s) to be tested (conditional)
-D filename of the upload description file (optional)
COM port switches: (optional)
-P COM port number [0-2/addr,irq] -L force local mode
-B DTE baud rate [300-115200] -H handshake [Hardware/Software/Both]
-1 disable 16550 FIFO operation -X use FOSSIL driver interface
-O use OS/2 SIO API interface
Miscellaneous switches: (optional)
-N BBS node number -G ANSI graphics toggle [Yes/No]
-R retain original archive date -T readability time delay [0-10]
-Q quiets beep on error -! enables debugging operation
ULPSM.EXE command-line syntax:
ULPSM -Cd:\path\config.cfg [-Ad:\path\<file.ext>] [-U] [-R] [-S] [-P#] [-I]
[-T#] [-Q]
-C complete path and filename of the configuration file
-A file(s) to add to the duplication database (optional)
-U forces unpacking of all archives when adding files (optional)
-R recurses nested drive subdirectories when adding files (optional)
-S compiles, sorts and indexes the duplication database (optional)
-P age in months for purging duplication database records (optional)
-I initialize process data file (optional)
-T sets time delay for display readability (optional)
-Q quiets beep when an error is detected (optional)